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DETAILED ACTION 
Response to Arguments 

1. Applicant's arguments with respect to claims 1-15, 17, 18, 20, 21, and 23-26 
have been considered but are moot in view of the new ground(s) of rejection. 



Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1-7, 9, 15, 21, 23, 24, and 26 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over U.S. Patent 6,282,713 to Kitsukawa et al. (Kitsukawa) in view 
of U.S. Patent 6,452,598 to Rafey et al. (Rafey). 

Regarding claim 1 , Kitsukawa teaches a tuner (fig. 3, label 21 ), a QPSK 
demodulator in communication with said tuner (fig. 3, label 22, col. 5, II. 18-28), wherein 
the tuner receives a broadcast signal including advertising data (claimed annotation 
data) (col. 5, II. 29-46), the advertising data including graphics data for overlaying 
graphic images on a frame associated with video objects (col. 6, II. 9-13, col. 8-9, II. 58- 
33), a video decoder in communication with said demodulator (fig. 3, label 25), a display 
device in communication with said demodulator (fig. 2, label 4, col. 5, II. 29-55), an 
SRAM memory for storing annotation data (fig. 3, label 36, col. 6, II. 5-13), a CPU (fig. 3, 
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label 29) in communication with said demodulator. Kitsukawa teaches storing a 
computer program that controls when executed by said central processing unit, a 
display of said annotation data on said display device (col. 5, II. 56-65). Kitsukawa 
teaches the use of SRAM (36), ROM (37), EEPROM (38), and SRAM (51 , 52), which 
equates to the claimed memory. Kitsukawa discusses showing the advertisements with 
the advertising, but Kitsukawa is silent on determining whether the object is visible in 
the video and controls display of the associated graphics responsive to a determination 
that the video object is visible in the video frame. In analogous art, Rafey teaches 
interfacing 3-D graphics content with broadcast video, wherein the system transmits a 
mask for the video to implement shaped video effects and the viewer device renders to 
make the appropriate regions visible to the user (col. 8, II. 39-50, which equates to 
determining whether the object is visible in the video and controls display of the 
associated graphics responsive to a determination that the video object is visible in the 
video frame. Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify Kitsukawa by determining whether the object 
is visible in the video and controls display of the associated graphics responsive to a 
determination that the video object is visible in the video frame as taught by Rafey in 
order to incorporate interactive graphics into the digital bradcast content to greatly 
enrich the viewers' experience (Rafey: col. 1, II. 46-50). 

Regarding claim 2, Kitsukawa teaches an integrated receiver/decoder (IRD), 
which equates to a viewer interaction device (fig. 2 and 3, label 2; col. 5, II. 1-28), 
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wherein Kitsukawa teaches the user selecting an advertisement mode, wherein the 
annotation data is displayed to the viewer (col. 6-7.ll. 65-21 ), which equates to 
displaying annotation data on said display device in response to a viewer activating said 
viewer activating said viewer interaction device (fig. 5-11). 

Regarding claim 3, Kitsukawa teaches selection of the advertisements (col. 8, II. 
17-57), but Kitsukawa is silent on the graphics image associated with mask information 
to identify regions on the display. Rafey teaches the graphics image associated with 
mask information to identify regions on the display (col. 8, II. 39-50). Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to modify Kitsukawa by the graphics image associated with mask information to identify 
regions on the display as taught by Rafey in order to permit interactive product behavior 
in a cost effective manner. 

Regarding claim 4, the combination of Kitsukawa and Rafey teach encoding 
object information with an object timelist (claimed object time stamp) (Rafey: col. 9, II. 
41-45). 

Regarding claim 5, the combination of Kitsukawa, Rafey, and Blackketter 
teaches a time stamp to display mask information in temporal relation to a video frame. 
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Regarding claim 6, Kitsukawa teaches storing object information to display 
objects in a video frame on the display (fig. 4, col. 6, II. 51-60), which equates to 
annotation data including object information associated with the video object. 

Regarding claim 7, the combination of Kitsukawa and Rafey teach encoding 
object information with an object timelist (claimed object time stamp) (Rafey: col. 9, II. 
41-45). 

Regarding claim 9, Kitsukawa teaches using a list of pointers (claimed object 
mapping table) for the addresses of advertising and coupon information, which equates 
to an object mapping table, wherein upon accessing the advertising and coupon 
information by the addresses in the table, the CPU can retrieve the data from the data 
buffer (51), which equates to an object property table referenced by said first identifiers 
(list of pointers), and said object property data including a first set of annotation data. 

Regarding claim 15, Kitsukawa teaches an IRD (fig. 2, label 2), receive (fig. 3, 
label 21), decode (fig. 3, label 25), store (fig. 3, labels 36, 37, 38, 51, 52) and 
manipulate broadcast information by displaying advertisement information, central 
processor unit (fig. 3, label 29) and at least one memory devices, receiving broadcast 
information via the tuner; decoding said broadcast information to recover graphics data 
information for overlaying a graphics image on a video frame, the image associated a 
video object (fig. 3, label 25, col. 6, II. 9-13, col. 8-9, II. 58-33). 
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Kitsukawa is silent on graphics data in a queue in said at least one memory 
devices. Official Notice is taken that the use of queues is well known in the art. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Kitsukawa by using queues in order to properly process 
data in order, thereby conserving processing power to locate specific elements. 

Kitsukawa discusses showing the advertisements with the advertising, but 
Kitsukawa is silent on determining whether the object is visible in the video and controls 
display of the associated graphics responsive to a determination that the video object is 
visible in the video frame. In analogous art, Rafey teaches interfacing 3-D graphics 
content with broadcast video, wherein the system transmits a mask for the video to 
implement shaped video effects and the viewer device renders to make the appropriate 
regions visible to the user (col. 8, II. 39-50, which equates to determining whether the 
object is visible in the video and controls display of the associated graphics responsive 
to a determination that the video object is visible in the video frame. Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to modify Kitsukawa by determining whether the object is visible in the video and 
controls display of the associated graphics responsive to a determination that the video 
object is visible in the video frame as taught by Rafey in order to incorporate interactive 
graphics into the digital bradcast content to greatly enrich the viewers' experience 
(Rafey: col. 1 , II. 46-50). 
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Regarding claim 21, Kitsukawa teaches receiving advertising and coupon 
information, which is received from the broadcast transmitter (col. 6, II. 5-18, col. 5, II. 
18-28), which equates to transmitting information in conjunction with a video signal for 
display in a temporal relation with the video signal. Kitsukawa teaches encoding object 
information (col. 6, II. 5-18) associated with the products and transmitting the objects 
with the video signal for display in a temporal relation with the video signal. Kitsukawa 
is silent on the timing mechanism used to display the information and encoding mask 
information and a visibility indicia, wherein the visibility indicia indicates whether one or 
more objects in an associated video frame are enabled for being visually identifies and 
displaying the mask upon determination that the objects are enabled to be visually 
identified. 

In analogous art, Rafey teaches interfacing 3-D graphics content with broadcast 
video, wherein the system transmits a mask and timing information (via a timelist) for 
the video to implement shaped video effects and the viewer device renders to make the 
appropriate regions visible to the user (col. 8, II. 39-50), which equates to encoding 
mask information and a visibility indicia, wherein the visibility indicia indicates whether 
one or more objects in an associated video frame are enabled for being visually 
identifies and displaying the mask upon determination that the objects are enabled to be 
visually identified. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to modify Kitsukawa by encoding mask a with 
timing information and a visibility indicia, wherein the visibility indicia indicates whether 
one or more objects in an associated video frame are enabled for being visually 
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identifies and displaying the mask upon determination that the objects are enabled to be 
visually identified as taught by Rafey in order to incorporate interactive graphics into the 
digital broadcast content to greatly enrich the viewers' experience (Rafey: col. 1 , II. 46- 
50). 

Regarding claim 23, the combination of Kitsukawa and Rafey teaches 
determination of that the video object is visible in the video frame based on the visibility 
indicia, wherein the indicia indicates whether the object is visible (Rafey: col. 8, II. 39- 
50). 

Regarding claim 24, Kitsukawa teaches decoding said broadcast information to 
recover graphics data information for overlaying a graphics image on a video frame, the 
image associated a video object (fig. 3, label 25, col. 6, II. 9-13, col. 8-9, II. 58-33). 
Kitsukawa is silent on graphics data in a queue in said at least one memory devices. 
Official Notice is taken that the use of queues is well known in the art. Therefore, it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to modify Kitsukawa by using queues in order to properly process data in order, 
thereby conserving processing power to locate specific elements. 

Regarding claim 26, the combination of Kitsukawa and Rafey teach encoding 
object information with an object timelist (claimed object time stamp) (Rafey: col. 9, II. 
41-45) and teach transmitting the encoded object information with said video for display 
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in temporal relation with said video signal (Kitsukawa: col. 6, II. 5-18, col. 5, II. 18-28, 
Rafey: col. 8, II. 39-50) 

4. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 6,282,713 to Kitsukawa et al. (Kitsukawa), U.S. Patent 6,452,598 to Rafey et al. 
(Rafey) in view of U.S. Patent 6,41 5,438 to Blackketter et al. (Blackketter). 

Regarding claim 8, Kitsukawa and Rafey are silent on indicating an expiration 
time. Blackketter teaches expiring triggers, which are indicative of the last instance the 
data structure is used (col. 3, II. 13-22, col. 10, II. 24-35). Therefore, it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify 
Rafey by indicating an expiration time in order to ignore invalid triggers (Blackketter: col. 
10, II. 24-35). 

5. Claims 10-14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 6,282,713 to Kitsukawa et al. (Kitsukawa) and U.S. Patent 6,452,598 to 
Rafey et al. (Rafey) in view of U.S. Patent 5,889,746 to Moriyama et al. 

Regarding claim 10 and 1 1 , Kitsukawa and Rafey are silent on the annotation 
data field is a title data field and the third data structure element is a string including the 
title of the object. Moriyama teaches a pointer to a text string wherein the text string can 
be the title (col. 15-16, II. 65-31; see also fig. 9). Therefore, it would have been obvious 
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to one of ordinary skill in the art at the time the invention was made to modify Kitsukawa 
and Rafey by pointing to a string including the title as taught by Moriyama in order to 
efficiently manage variable length fields and manage the memory of the system. 

Regarding claim 12, Kitsukawa teaches menus with display identifiers and 
actions associated with the display identifiers (fig. 5, col. 17-57). However, Kitsukawa is 
silent on an annotation data menu field and the second identifier referencing a selector 
including a set of display identifiers and corresponding actions. Moriyama teaches 
identifying elements via pointers, as shown in figure 9. Therefore, it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify 
Kitsukawa and Rafey by identifying a display identifiers and actions via pointers as 
- taught by Moriyama in order to efficiently manage dynamic resources. 

Regarding claim 13, Kitsukawa and Rafey are silent on never duplicating the first 
and second identifiers. Official Notice is taken that never duplicating identifiers is well 
known in the art. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to modify Kitsukawa and Rafey by never 
duplicating identifiers in order to appropriately reference the appropriate information 
thereby eliminating access to the incorrect data structure. 

Regarding claim 14, Kitsukawa and Rafey are silent on a variable value. Official 
Notice is taken that a variable values are well known in the art. Therefore, it would have 
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been obvious to one of ordinary skill in the art at the time the invention was made to 
modify Kitsukawa and Rafey by using variable values in order to efficiently manage the 
memory by dynamically referencing and allocating the memory. 

6. Claims 17, 18, 20, and 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 6,282,713 to Kitsukawa et al. (Kitsukawa) and U.S. 
Patent 6,452,598 to Rafey et al. (Rafey) in view of U.S. Patent 6,317,774 to Jones et al. 
(Jones). 

Regarding claim 17, the combination of Kitsukawa and Rafey teaches mask 
information, which is decoded along with comparing the time information in order to 
display the information, but Kitsukawa and Rafey are silent on the assigning threads to 
perform specific tasks. Jones teaches a thread scheduler for execution of real-time and 
non-real-time tasks (col. 4, II. 27-57), wherein the scheduler can be implemented in a 
set top box (col. 6, II. 37-41 ), which teaches assigning by the scheduler different threads 
for performing tasks. Therefore, it would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify Kitsukawa and Rafey to assign 
threads to perform specific tasks as taught by Jones in order to use a thread to decode 
information, along with a thread for comparing data for the benefit of increasing the 
efficiency of the system in addition to effectively schedule non-real-time programs in the 
complete absence of real-time program, reservations and constraints (Jones: col. 2, II. 
47-50). 
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Regarding claim 18, Kitsukawa is silent on each mask having a time stamp, 
comparing the time stamp with a time stamp of a displayed video frame; and displaying 
said mask based on a relationship between said mask time stamp and said time stamp 
of said displayed video. Rafey teaches that the masks have a timelist (mask having a 
time stamp) and a timecode (claimed time stamp of a displayed video frame), wherein 
when a match occurs, the script nodes triggers behavior changes of objects in the 
scene. Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Kitsukawa by comparing the time of display to 
the actual display and displaying the mask accordingly as taught by Rafey in order to 
display information at the appropriate time while providing additional services and 
interactivity to the user. 

Regarding claim 20, Kitsukawa is silent on processing a mask time stamp 
corresponding to a frame not yet displayed. Rafey teaches decoding the mask, which is 
inherently placed in a buffer in order to process the mask, Rafey teaches determining if 
the mask should be displayed and displaying the information (col. 9-10, II. 59-4), 
however Rafey is silent on sleeping for a time calculated to end at the time the mask is 
to be display. Official Notice is taken that determining the difference in time to perform 
an event is well known in the art. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kitsukawa and 
Rafey by determining the difference in time to perform an event in order to efficiently 
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manage events thereby reducing the processing of the system by selectively querying 
the system clock. 

Regarding claim 25, Kitsukawa teaches a SRAM containing advertisement and 
coupon information (col. 6, II. 5-18), but Kitsukawa and Rafey are silent on the use of 
threads, consequently, Kitsukawa and Rafey are silent threads capable of adjusting a 
behavior of mask data and a second thread capable of adjusting behavior of object 
data. Jones teaches a thread scheduler for execution of real-time and non-real-time 
tasks (col. 4, II. 27-57), wherein the scheduler can be implemented in a set top box (col. 
6, II. 37-41), wherein the thread of Jones is inherently capable of adjusting a behavior of 
mask data and a second thread capable of adjusting behavior of object data, in that a 
thread is capable of any processing task performed by the processor. Therefore, it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to modify Kitsukawa and Rafey by using threads capable of adjusting a behavior 
of mask data and a second thread capable of adjusting behavior of object data as 
taught by Jones in order to process plural tasks simultaneously in a real time 
environment thereby increasing the efficiency of the system in addition to effectively 
schedule non-real-time programs in the complete absence of real-time program, 
reservations and constraints (Jones: col. 2, II. 47-50). 



Conclusion 
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7. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Andrew Y. Koenig whose telephone number is (571) 

272- 7296. The examiner can normally be reached on M-Th (7:30 - 6:30). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Christopher Grant can be reached on (571) 272-7294. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 

273- 8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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